Maximum Entropy Models and Prepositional Phrase Ambiguity

Prepositional phrases are a common source of ambiguity in natural language and many approaches have been devised to resolve this ambiguity automatically. In particular, several different machine learning approaches have now reached accuracy rates of around 84.5% on the benchmark dataset. Maximum entropy (maxent) models, despite their successful application in many other areas of natural language processing, have only reached 83.7% accuracy on this task. This dissertation shows that maxent models can achieve accuracy rates of 84.7% using standard features, rising to 85.3% when using additional features based on Latent Semantic Analysis (LSA) information. Three feature selection techniques are compared in this domain: frequency cut-off, information gain (mutual information) and a new method that uses the variation of feature weights across several training sets. A simple frequency cut-off is found to be the most robust method across of a variety of models. We also consider ensembles of maxent classifiers created using bagging and noisy bagging, a more effective variant where additional random noise is added to each training set. The latter approach creates a more diverse set of classifiers and a maxent ensemble using noisy bagging achieves 85.53% accuracy.